AN EXTENSION AND TRACE THEOREM FOR FUNCTIONS OF 
H-BOUNDED VARIATION IN CARNOT GROUPS OF STEP 2 

CHRISTINA SELBY 



Abstract. This paper provides an extension of a function u £ BVh(Q) 
to a function uq £ BVh(G), when Q C G is "H-admissibile," and G is a 
step 2 Carnot Group. It is shown that H-admissible domains include non- 
characteristic domains and domains in groups of Heisenberg type which have 
a partial symmetry about characteristic points. An example is given of a do- 
main f2 that is C 1,a , a < 1, that is not H-admissible. Further, when O is 
H-admissible a trace theorem is proved for u E BVh (H) . 

1. Introduction 

There has been significant progress in the study of functions of H-bounded vari- 
ation and sets of finite H-perimeter in such works as |SJ ^fl EH ■ This 
paper utilizes these results to further this progress. In particular, the structure 
theorem for sets of finite H-perimeter in step 2 Carnot groups, proved by Franchi, 
Serapioni, and Serra Cassano in |12j . provides a way to define an "H-admissible 
domain." Their result states that for a set E with finite H-perimeter measure, the 
reduced boundary of E, d^E, is H-rectifiable. Further, the H-perimeter measure 
of E is the (Q — l)-spherical Hausdorff measure with respect to d restricted to 
dftE, where d is a distance equivalent to the Carnot-Caratheodory distance. This 
measure is denoted These H-admissible domains are significant, for if il is 
H-admissible and it G BVh{Q), one may extend u to be zero outside of ft and ob- 
tain a function ua in BVh(&). The H-admissibility condition given is an analogue 
to the condition stated in Meyer's and Ziemer's work in 15 . The definition of 
H-admissible is given below, where 3*,hE is the measure-theoretic boundary of E: 

Definition 1.1 (H- Admissible Domain). A bounded domain of finite H-perimeter 
is said to be H-admissible if the following two conditions are satisfied: 

(i) s$- 1 (dn\d* >H n) = o. 

(ii) There is a constant M = M(VL) such that for each x G dfl there is a d-ball 
B(x,r) with 

Sd~\ d *,HE n d* tH Q) < M S^\d^ H E n Q) 

for all E cTir\B(x,r). 

Using Meyer's and Ziemer's Euclidean definition, Lipschitz domains can be shown to 
be admissible quite easily using the Gauss-Green theorem. However, a wide class 
of H-admissible domains in the setting of step 2 Carnot Groups is not so easily 
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obtained. This paper shows that H-admissible domains include non-characteristic 
domains and domains in groups of Heisenberg type which have a partial symmetry 
about characteristic points. In proving condition (i) of H-admissibility a fact proved 
by Magnani in ^] is vital. It states the measure of the characterstic set of a C 1 
domain in a step 2 Carnot group has S^ 1 measure zero. This result is an extension 
of a result by Balogh for the Heisenberg group ;<C ■ This paper also gives an example 
of a C 1,a domain, a < 1 that is not H-admissible. 

The extended function uq is not defined on dil, but this paper defines the trace 
of u on dfl, u* , as the sum of the upper and lower approximate limit of uq, fi 
and A, respectively. Here, the definition of upper and lower approximate limit 
is that of Federer The set F, where the lower approximate limit is strictly 
less than the upper approximate limit is shown to be H-rectifiable. Ambrosio and 
Magnani use a stronger definition for approximate continuity in [3], and prove that 
the approximate discontinuity set is H-rectifiable. This paper shows that the set 
F, however, has some additional useful properties. The stated rectifiability result, 
along with an implicit function theorem proved in |12| lead to the following trace 
theorem: 

Theorem 1.2. If is an H-admissible domain, there is a constant M ~ M(fi) 
such that 



[ \u*\ d\dQ,\ H = C(n + m) [ \u*\ dS®' 1 < M 



U \\BV H {£i) 

whenever u E BVh (^)- 

The additional useful properties of the set F are stated below: 

Theorem 1.3. Let u £ BV H (&), then 

(i) F is countably H-rectifiable, 

(ii) — oo < X(x) < < oo for S®" 1 a.e. x E G, 

(iii) for S? 1 a.e. y E F, there is a vector v{y) such that va s {u) = v(y) 
whenever X(y) < s < /i(y). 

(iv) For all y as in (iii), with —oo < X(y) < fi(y) < oo there are Lebesgue 
measurable sets F + and F~ such that 



lira 



= lim 



\F-nT y (S H (v(y)))nB(y,r)\ 
" \T y {S- H {v{y)))CM3{y,r)\ 

\F+nT y (s+(v(y)))nB(y,r)\ 



^ \r y (S^(y)))nB(y,r)\ 



and 



lim u{x) = n(y), lim u{x) = My). 

xeF-nr y (S-( V {y))) xGF + nr v (S+^(y))) 

In (iv), the sets S^(i/(y)) and {v(y)) are the half-spaces obtained in the blow- 
up theorem found in [T5|. A Poincare inequality proved by Garofalo and Nheiu in 
[T3| for functions in BVh(B p (x, r)), B p (x, r) a gauge ball, along with part (iii) are 
used to prove some integral properties of a function u E BVh{G). These are stated 
below, where U(x) is the average of the upper and lower approximate limit of u: 

Theorem 1.4. Assume u E BVh(<&). Then 
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(i) linv^o i Bp{x r) \u - U(x)\Q^ dh = for 1 a.e. x e G\F, and 

(ii) for S3 1 a.e. x G F . there exists a vector v = v{x) such that 

lim + \u - ^(x)l^ 1 dh = 0, 

r ^iB,(i,r)nS fl M 

and 

lim / |u- A(x)|^ d/i = 0. 

Using the definition of Ambrosio, Fusco, and Pallarain for approximate jump 
point, one can then observe that a function u G BVh{G) has an approximate jump 
point for S^ 1 a.e. point in F. 

2. Notation and Preliminary Results 

A Carnot group G of step 2 is a simply connected Lie group whose Lie algebra 
admits a step 2 stratification. This means there are subspaces V\ , V 2 such that 

S = V 1 ®V 2 , [V l ,V 1 ] = V 2 , [V u V 2 }=0, 

where [Vi,T4] is the subspace of g generated by the commutators [X, Y], where 
X G Vi, Y G Vfc. Throughout this paper, let m — dim(Vi), and n = dim(T4). 
Let {ei, . . . , e m } denote an orthornormal basis for Vi, and {ei, . . . , e„} denote an 
orthonormal basis for V 2 . Let X — {Xi, . . . ,X m } be the family of left-invariant 
vector fields where Xi(Q) = a. The set {X\. .... X m } along with all of its commu- 
tators generates g. The exponential map is known to be a global diffeomorphism, 
and using it, one is able to identify an element of G with an element of M. m+n . In 
particular, for p G G, p = exp(piAi + • • • + p m + m A" m + n ), and can be identified 
with (pi, . . . ,p m+n ) G R m+n . Thus G can be identified with (R m+n , •), where the 
group operation • is determined by the Campbell-Hausdorff formula. 

The horizontal bundle HG is the subbundle of the tangent bundle TG that is 
spanned by Xi, . . . , X m . The fibers of HG are 

HG X = span{Ai(a;), . . . , X m (x)}, x G G. 

Each fiber of HG is endowed with a scalar product (•, -) x and a norm | • \ x that makes 
the basis {X±(x), . . . , X m (x)} an orthonormal basis. So for p — Y^i+i Pi^i( x ) = 
(pi, . . . ,p m ), and q = Yh+i <li X i( x ) = ■ • • > Qm), (p, q) = YnLxPilh an d \p\ 2 x = 
(jp,p) x . The sections of HG are called horizontal sections and a vector in HG is 
called a horizontal vector. A horizontal section <fi can be identified with a function 
<f) = (0i, . . . ,4> m } ■ K m+ " — » K m by identifying it with its canonical coordinates 
with respect to {X\(x), . . . , X m (x)}. The notation (</>, ?/;) represents (</>(a:), ip{x)) x . 
The horizontal divergence of : R m+ " — > R m is defined as 

m 

div H 0(a;) := }^Xi4>i(x), 

i—l 

and the horizontal gradient of it G C 1 (G) is defined as 

V H u(aO := (Xiu(x), . . . , X m u{x)). 
Translations and dilations are defined on G as follows: 

r P (q) =p-q, S r (p) = (rpi, . . . ,rp m ,r 2 p m+1 , . . .,r 2 p m+n ). 
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Then Q :— to + 2n is called the homogeneous dimension of G. 

The results of Franchi, Serapioni, and Serra Cassano utilize the distance defined 
below: 

d(x,y) = d{y~ 1 ■ x,0), 
where for p = (p x , . . . ,p m+n ) € W n+n , 

1 /2 

(2.1) rf(p,0) = max{||(pi, . . . ,p m )|| H m, e\\(p m+1 , . . . ,p m + n )\\^ }■ 

Here e G (0, 1) is a constant depending on the group structure. Throughout this 
paper, B(x,r) := {y € G : d[x,y) < r}. This distance <i is known to be equivalent 
to the Carnot Caratheodory distance, and thus also equivalent to the gauge distance 
(see 521)- The gauge distance is given below: 

p{x,y) = piy^ 1 ■ x,0), 

p{x, 0) = {{x\ + ■■■ + x 2 m f + x 2 m+1 + ■■■ + x 2 m+n ) 1/4 . 

The gauge balls B p (x, r) := {y £ G : p(x, y) < r} are known to be PS-domains, see 
|13j . for which the isoperimctric inequality and a Poincare inequality hold. These 
inequalities will be stated shortly. The (Q — l)-dimcnsional spherical Hausdorff 
measure with respect to d will be denoted Proposition 2.4 in 2] states that 

the diameter of B(x,r) with respect to d is 2r. 

The H- variation of u £ Lj oc (£l) with respect to £1 is given as 

Vara(u;n) = sup{ / udiv H <t>dh : £ C£(fi;R TO ), \</>(P)\ < 1}, 

JQ. 

where dh is (to + n)-dimensional Lebesgue measure on R m+ " The function u is 
said to belong to BVh{£1) if Vara(u; fl) < oo. The H-perimeter measure of SI with 
respect to the set E is given as \dE\n(Q) := Varn(x£; fi). The set S is said to have 
finite H-perimeter, or be H-Caccioppoli, if |<9-E|#(G) < oo. By Riesz representation 
theorem is a Radon measure on G and there exists a measurable section ve 

of H& such that 

/ div H 0d/i = - / ((j>,v E )d\dE\ H , 

J E JG 

for any <j> £ Cq(G, HQ). The section is called the generalized inward normal to 
E. The following proposition will be used in a following section and is proved in 


Proposition 2.1. If E is H-Caccioppoli with Euclidean C 1 boundary, then 

« m 

\dE\ H {n) = / (J2(x t ,n)l m ) 1/2 dn m+n - 1 , 

JdEnn i=1 

where ■}{ m + n ~ 1 { s Euclidean (to + n — 1)- dimensional Hausdorff measure and n = 
n(x) is the Euclidean unit outward normal to dE. 

If CI can be described as fl = {4> < 0}, then <f> is called a defining function for O. 
Further, n = — V0/|V0|. The above proposition will be used with such a defining 
function. 

The following three theorems are proved in Theorem 12.41 was also proved 
independently in |ll)j . 
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Theorem 2.2. There is a positive constant c > such that for any H-Caccioppoli 
set E, for all x £ G, and r > 0, 

min{\E n B p (x, r)\, \E C n B p (x, r)]} 9 ® 1 < c\dE\ H (B p (x,r)), and 

mm{\E\,\E c \}^ <c\dE\ H (G), 
where \E\ is the (m + n)- dimensional Lebesgue measure of E. 

Theorem 2.3. For any u £ BVh(B p (x, r)), one has C = C(m + n) such that 
\\u — U xr \\ < CVarH(u:Q), 

' L^T^T (B p (x,r)) 

where u x>r denotes the integral average of u over B p (x,r). 
Theorem 2.4. (Coarea Formula) Let u £ BVh{Q)- Then 

/oo 
\dA t \ H {Q.)dt, 
-oo 

where At — {x £ ^ : u(x) > t}. 

Now for some measure-theoretic defintioris. 

Definition 2.5 (Reduced Boundary). Let E be an H-Caccioppoli set; then x £ 
dftE, the reduced boundary of E, if 

\dE\ H (B{x, r)) > for any r > 0, 
there exist lira +■ i>e d\dE\u, and 

r ^ Q JB(x,r) 

lim J- v E d\dE\ H = 1. 

r ^°J B(x,r) 

Definition 2.6 (Measure-Theoretic Boundary). Let E c G 6e a measurable set, 
then x £ d*^nE, the measure theoretic boundary of E, if 

\EnB(x,r)\ „ , |£ c n 5(a;,r)| „ 

lim sup 1 , , . > and lim sup 1 , . \ , ' > 0. 
r ^o+ |-B(a;, r-)| r ^ + \B{x,r)\ 

The following lemma and theorem can be found in 12 : 
Lemma 2.7. Let E be an H-Caccioppoli set, then 

(i) d* H E c C 

(ii) Sf- 1 (d^ H E\a* H E)=0. 

Theorem 2.8 (Gauss-Green Theorem). Let E be an H-Caccioppoli set, then 

dW H cf>dh = 8 d [ (v E ,(f))dS$~\ for all cj) £ C%(G, HG). 

Jd,, H E 

The following definition, found in ^2] is needed for the statement of the implicit 
function theorem. Here C^{Q) is the collection of functions u with distributional 
derivatives Xiu that are continuous in fi. 

Definition 2.9. S C G is an H-regular hypersurface if for every x £ S there exists 
a neighborhood U of x and a function f £ C^U) such that 

Sf\U = {y£U: f{y) = 0}, and 

Vh/(») ^0 fory£U. 
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Theorem 2.10 (Implicit Function Theorem). Let £1 be an open set in M. m+n , G fi 
and let f G Cjj(D,) be such that Xif(0) > 0, /(0) = 0. Define 

E = {x G Q : f{x) < 0}, S = {x G Q : f(x) = 0}, 

and, /or 5 > 0, h > 

h = {£ = (6, ■ ■ .,fm+n) G K m+ " _1 , fol < (5}, A - [h, k]. 

If £ = (£2, • ■ ■ ,£,m+n) G K m +™ _1 arid £ € Jh, denote by y(t,£) the integral curve of 
the vector field X\ at the time t issued from (0,£) G R m+ ™ ; i. e. 

Then there exists 8, h > such that the map (t, £) — * 7(f, £) is a homeomorphism 
of a neighborhood of Jh X Is onto an open subset ofM. m+n , and, if hi CC is the 
image of Int{Jh X 7a) through this map, 
(i) /ias finite H -perimeter in IA; 

(h) dEnn = snU; 

(iii) i^e(s) = — V Hf(x)/\V Hf(x)\ x for all x G SOU, where ve is the generalized 
inner unit normal. Moreover, there exists a unique function 

<f> = </>(£) : I s -> J h 

smc/i i/iai i/ie following parameterization holds: if £ £ Is and we put $(£) = 

Snfi = {a;eM:i = $(0, £ € 
is continuous] 
the H-perimeter has an integral representation 



^ H{u) -] h — xjim — 

Theorem 2.11 (Structure Theorem). If E C G is an H-Caccioppoli set, then 
dflE is (Q — 1) -dimensional H-rectifiable, 

that is, d* H E = N U |J£Li Kh, where H^ 1 (N) = and K h is a compact subset of 
a H-regular hypersurface Sh; 

ve{p) is H-normal to Sh at p, Vp G Kh, 

\dE\ H = d d S^- 1 [d* H E, 



wh< 

e d = ^ 

UQ-l WQ_i 



d = CJm - lCJ " £ " - -±—H m+n - l (dS+(y E {U) n B(0, 1)). 



Here e is as in \2.1\ 

Note that by lemma l2"71 the measure theoretic boundary of E is also H-rectifiable. 

The half-spaces S^{ve{P)) and S^(ve{P)) appear in the blow-up theorem 
stated in theorem 3.1 of ^21- The hyper-planes can be thought of as "approxi- 
mate tangent planes" to the boundary of E at p. They are defined below: 

s h{"e(p)) ■= U ■ (npq,VE(p)) P > 0}, 
s h(ve(p)) ■= {q ■ (Tr p q,VE(p))p < 0}, 
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where for q = (qi, . . . , q m , q m +i, ■ ■ ■ , Qm+n), 

m 

v P q = « = • • • » 9m, o, . . . , 0) e M m+ ". 

i=l 

The reader may have noticed that the set where the "measure-theoretic normal" 
exists has not been explicitly defined. This set is often defined in classic geometric 
measure theory texts. There is no need to distinguish this set in this work because 
of lemma YI7I\ and the following lemma which is lemma 3.3 in |12) : 

Lemma 2.12. Let p e d* H E. Then 

y \B(p,r)nEnT p (Sx(Mp)))\ _ n 

\B(p,r)\ 

hm — r-: = 0, and 

r^O \B(p,r)\ 

l^ug( P) ,)) = 

i — >o 

The right-hand side of the last statement of the lemma is actually a constant, 
as can be seen in theorem 3.1 of |12j . 



3. Examples of H-admissible domains 

In the following sections, a function u £ BVh{Q) is shown to have an extension 
in BVh(Gi) when SI is H-admissible. This section provides examples of H-admissible 
domains, and proves that if fl is a C 1,a domain, a < 1, then it is not necessarily 
H-admissible. The definition of an H-admissible domain follows: 

Definition 3.1 (H- Admissible Domain). A bounded domain Q of finite H-perimeter 
is said to be H-admissible if the following two conditions are satisfied: 

(i) 5^ 1 (ao\s*, ff n) = o. 

(ii) There is a constant M = Af(fi) such that for each x S 9S1 there is a ball 
B(x,r) with 

S^\d^ H End,M^) < M S^\d^ H Er\n) 

for all E C Hn B(x,r). 

Before stating any theorems about H-admissible domains, a few observations 
should be made. In the classical setting, it is proved that Lipschitz domains are 
admissible using the Gauss-Green theorem by setting V = (1,0) and utilizing the 
nice form of the normal vector of a graph (see Remark 5.10.2 in ^D). Observe that 
if n is a C 1 domain with no characteristic points then without loss of generality, 
for x S dft, there is a ball B(x, r) such that z^n,i > K > in dft n B(x, r). Taking 
V = (1,0), one obtains 

f {v n ,V) dS®- 1 = f (u E , V) dS®- 1 , 

for E C 51 D B(x, r), which implies 

KS^-^d^HEnd^H^) < s i fl-1 (a,,HBnn). 
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By the compactness of dfl a covering argument gives a uniform K such that the 
above holds. Since f2 is C , it has finite H-perimeter. The following proposition, 
found in |14) provides condition (i) of H- admissibility for C 1 domains: 

Proposition 3.2. For il C G a C 1 domain, the characteristic set of fl is S^ 1 - 
negligible. 

Since a C 1 domain has a horizontal unit normal at all points except characterstic 
points, (i) holds. This proves the following proposition. 

Proposition 3.3. A bounded C 1 domain O C G, with no characteristic points is 
H- admissible. 

Since H-admissibility is a local condition, the proof of the previous proposition 
reveals that condition (ii) of H-admissibility will hold true for C 1 domains at points 
outside a neighborhood of each characteristic point. Further, proposition |^21 pro- 
vides condition (i). Thus, to prove a domain is H-admissible it is sufficient to show 
there is M(Q) > such that for every characteristic point x 6 d£l, there is r > 
such that 

s^- l {d^ H E n d*, H Q) < M S^id^aE n fi) 

for all E C f2 H B(x,r). The next proposition gives a condition for domains in a 
group of Heisenberg type to be H-admissible when the domain a partial symmetry 
about characteristic points. First, some definitions are needed. Recall from sec- 
tion 2 that {ei, . . . , e m } and {ei, . . . , e„} denote orthonormal bases for V\ and V% 
respectively. 

Definition 3.4. In a Carnot group G of step 2 with Lie algebra Jj = Vi © V2, the 

linear mapping J : V% — > End(Vi) is defined by 

Definition 3.5. A Carnot group G of step 2 is called of Heisenberg type if for 
every rj € V2, such that \rj\ — 1, the map J(r]) : V\ — * V\ is orthogonal. 

From the definitions, the following observations can be made for a Carnot group 
of Heisenberg type: 

(3.1) \Jm\ = \v\m, veV^teVu 



(3.2) {J(v)Z,Q = o, v&V 2 ^eV 1 , 

(3.3) (Jm,J(ri')0 = (r),v')\tf, V,v'eV 2 ,^eV 1 . 

Formulas for the vector fields Xi, i = 1, . . . , m have been obtained in [J] and 
|12j . They are as follows: 
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where b\j = ([e;, ej], ei). Using these formulas, the horizontal gradient of a function 
4> € C 1 can be written using the mapping J. Observe 

i=i 1=1 d 
dcj) 1 ^-A /r , , d(j) 

9<^> 1 ... . . 

where £ = i e and ^ = Ya=i §^ e i e ^- Therefore, 

(3.4) V^ = D^+i ,/(*?)£, 

where is the standard Euclidean gradient in K m . Thus, using 1X11 

(3-5) \V H <P\ 2 = \D,\ 2 + (D e , J(rj)0 + Jm 2 ^ 2 . 

So if <p is the defining function for a domain O, then 

(3.6) v a 



A notion of a domain being cylindrically symmetric about characteristic points 
is defined in [oj. A similar notion is defined in this paper: 

Definition 3.6. Let il C G be a bounded, connected, C 1 domain. Assume for each 
characteristic point P , after a group translation that sends P to the identity e = 0, 
one can find a neighborhood U of e such that 

(i) dil n U = {y n = -g(s, y%,..., y„-i)} n U, 

(ii) there exists M(fl) > such that §|| < M\s\ in U , 

(iii) 5(0,2/1, • ■ .,2/n-l) = §1(0,1/1, • ■ - ,2/n-l) = 0. and 

(iv) 5 is C 1 with respect to the variables y\, . . . ,y n —i- 

Here, Cl = P _1 il, s = \J x\ + ■ ■ ■ + x^ n = |£|. Then fl is said to have partial 
symmetry near its characteristic set. 

Theorem 3.7. Assume that C G has partial symmetry near its characterstic 
set. Then fl is H-admissible. 

Proof: Note that the H-admissibility condition is invariant under group translation. 
So without loss of generality, consider a domain £1 with characteristic point at 
e = G M m . Taking <j) = y n + g{s,y%, . . . , y n —i) as the defining function in the 
neighborhood U of and using EPA one computes 



s d 

vn — 



2 + i|??| 2 s 2 
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Here^ = Er=i 1 fr I + 6 n eV 2 . 

Let 5(0, r) C C/, and E C B(0, r) n Jl Consider the vector field V = This 
vector field is not Cq (G; i/G), and so it can not be used directly in the Gauss-Green 
theorem. However, the theorem can still be used to show H-admissibility. Define 



V S 



■^H i{ s> 6, 
-4^ if * < ^- 



V s is continuous, but not C^(G,HG). Let K : W n -> M, = if*(s), be a 

standard mollificr and define 



-YZi J{en) T KT ti ifs >^ 



where * is the standard convolution operator. Then Vs yT converges uniformly to Vs 
on compact subsets of G as r goes to 0. Further, multiplying Vg, T by a cutoff func- 
tion which is identically 1 in a neighborhood of E gives a vector field in Cq (G; HG). 
Call this vector field V$, T as well. 

Claim: divjjVj iT = on E. First observe that since Va jT does not depend on the 
second layer, the horizontal divergence is actually the standard Euclidean diver- 
gence in W 71 . Therefore, using integration by parts, the following is obtained: 



V S ,r (0 = / Vs.r (0 K T (£ - dl SO, 

divV 5lT (0= / (V«, T (|),D£(Jif T K-0))de 



7b(0,5) d 



J 



J{en) t, Di (K T ((;-0))d£ 



\B(0,S) |£| 



\ j-um — 1 



9B(0,<5) 



JdB(0,S) 6 

Now applying the Gauss Green theorem with V^ T one obtains 

/ (m,V s , T ) dS?' 1 = [ {v E ,V 5 AdS%-\ 



One can easily check that the right hand side is less than or equal to 1 {8*,hE n O) , 

a compact set, t 



for every r > 0. Since 8*.hE n is a compact set, the left hand side converges 

to 



a, H End, H f 
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as r goes to 0. To compute this integral, it is split as 

Jd riH End« )S nn{s>6} s 

( c — , u Wdb d 



9, |ff Bna,, H nn{s<<5} 
= h + Ih- 

Using 1331 13.21 1^751 and 13.61 one can compute 

>^n) Vjj0 = ( ,^ — ) + ( ^JinK, 

s s os s s 2 

s 

= 2' 



and 



Therefore, 



2 _ / 9 d\ , !, |2| C |2 



>d,, H End, iH nn{s>8} 



Observe that the integrand is in L°°(d* i HE D <9*,h^) by the hypothesis. In par- 
ticular, there is L(O) > such that 1 < |7y| 2 = Y^i=i {§§^j + 1 < £ since g is 
C 1 in the variables of the second layer. Let S go to 0. By Lebesgue dominated 
convergence Is — > I, where 

1 



HA)' 



Q-i 



Then by the hypothesis and above comment, 



/> . 1 =s%- 1 (d,,HEnd,, H n). 
Up + \l 



Similarly, one may compute 



I h = \ I . dSf- 

J d ,, H Ena,, H nn { s<s } ^ ~y> ~ , ~ 



By Lebesgue dominated convergence, iTa — > as <5 — > 0. Combining all of this 
information gives 



□ 
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The above theorem implies the H-admissibility of the gauge ball in the Heisenberg 
group. 

An example is presented showing that the condition SI g C 1,a , a < 1, is not 
sufficient for H-admissibility. Therefore, the assumption SI S C 1,a , a < 1 is not 
sufficient for H-admissibility for arbitrary Carnot Groups of step 2. 

Example 1. Assume SI C M , and <9S! is given by t = 1 — s 2 ~ e in a neighborhood 
of P = (0,0, f). One can compute that 



g'{s) 




2-e 


s 




s e 



and observe that g is not C ' in a neighborhood of P. For every B(P,1/N) a 
set En C B(P,r) n SI exists such that condition (ii) of H-admissibility fails. In 
particular, 



Consider A 



N 



{t > 1 - l/N}, and let E N 



N 



An n SI. First, observe that 



1 — s 2 e = 1 — l/N implies that s = (l/N) 2 -* . A defining function f, for An is 
f(s,t) — l/N — t. Therefore, by the structure theorem, 

s®- l {d*, H E N nfi) = s^^.hAn n fi) = c „ \dA N \ H (n). 

By vrovosition \2. U one computes 

\V„f\ dU 2 



\dA N \ H (Sl) 



d,, H A N C\Q, 



y/\x\i + \y\ 2 <(l/N)2- 



1 \V H f\dxdy 



2tt 



7T / 1 



Next, observe that 

s^-\d^ H E N n = s^-^b^hH n ^) = c„ |fln| H (^). 

TTie defining function for fl is g(s,t) = t + s 2 ~ c — f . TTius 



2tt 



V'|a:| 2 + |y| 2 <(l/JV) 



(2 - efr^ l -^> + — dr. 



F(x) 



r\/(2- e) 2 r 2 ^) + — dr. 



EXTENSION AND TRACE THEOREM FOR H-BV 



13 



Then in order to show f2 is not H- admissible it suffices to show 

lim F(x) = oo. 

x^O 

Using L 'Hospital's rule, one observes 



11/ 2(1-0 x 2 -- 

lim F(x) = lim - j \ (2 - efx ^ + — 

x^O x^O J x 2 - 1 V ^ 



4. Approximate Continuity and Functions of H-Bounded Variation 

Now that the class of H-admissible domains has been shown to be non-empty, 
this section provides needed tools to prove an existence and trace theorem for 
u G BVh{Q) : when Q, is H-admissible. 

The upper approximate limit of u at x, /J.(x), and the approximate lower limit 
of u at x, X(x), are defined below: 

H(x) := aplimsupu(y) = inf{i : D(A t ,x) = 0}, 
X(x) := apliminfu(y) = sup{£ : D(B tl x) = 0}, 

y—>x 

where At — {x G G : u(x) > t}, Bt = {x G G : u(x) < t}, and D(Q,x) is the 
density of fl at a; with respect to the balls B(x,r). The function u is said to be 
approximately continuous at x if X(x) = fi(x) = u(x). The following proposition is 
proved as in remark 5.9.2 of |16|. 

Proposition 4.1. The function u is approximately continuous at x if and only if 
there exists a Lebesgue measurable set E containing x such that D(E,x) — 1 and 
u[E is continuous. 

A very significant result in the classical theory is that a function of bounded 
variation is approximately continuous except for a set of 7Y n_1 -measure zero, where 
it has a "measure-theoretic jump," see |16| . A similar statement is true for functions 
of H-bounded variation with Euclidean Hausdorff measure replaced with 
Throughout the rest of this discussion, F will denote the following: 

F = {x G G : AO) < fi(x)}. 

Theorem 4.2. Let u G BV H {&), then 

(i) F is countably H-rectifiable, 

(ii) — oo < \(x) < n(x) < oo for 1 a.e. x G G. 

(iii) for S® 1 a.e. y G F, there is a vector v{y) such that VA s (y) = v {v) 
whenever \(y) < s < (J,(y). 

(iv) For all y as in (iii), with — oo < A(y) < n(y) < oo there are Lebesque 
measurable sets F + and F~ such that 

\F-nT y (S H (v(y)))nB(y,r)\ 



lim 



lim 



o \T y (S H (is(y)))nB(y,r)\ 

F+nT y (S+(v(y)))nB(y,r)\ 
;5 \T y (S+(y(y)))nB(y,r)\ 
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ana 

Jim u(x) = fx(y), lim u(x) = X(y). 

xeF-nr v (S H (u(y))) xeF+nr v (S+(u(y))) 

There are a few lemmas that will be used for the proof of the preceeding theorem. 

Lemma 4.3. There exists C — C(m + n) such that if J is a Borel set such that 
J C dtfE, then 

S%-\j)<C\dE\ H {J). 

Proof: Using lemma 12. 121 the proof of lemma 3.2.1 in \nj\ can be followed to reach 
the conclusion. □ 

Lemma 4.4. Let m + n > 1 and < r < 1/2. Let E be a Lebesgue-measurable set 
such that 

v \B(x,r)nE\ 

lim — — — rr — > t, when x £ E. 

r^o \B(x,r)\ 

Then there exists C = C(t, m+n) and sequence of gauge balls B p (xi, rj) with Xi G E 
such that 



Ec{jB p (xi,ri), 

i=l 

oo 

E^) Q - X < C \dE\M 



i=l 

Proof. First observe that if the hypothesis of the lemma holds with B(x, r) then it 
also holds with the gauge ball B p (x,r), except with r replaced with r', where r' 
also satisfies < t' < 1/2. To see this, first observe that since there is C > 1 such 
that 

C -1 p(^)y) < d(x,y) < C p(x,y), 
B p {x,r) C B{x,Cr). There are constants d > 1 and C p > 1 depending on the 
corresponding doubling constants of the respective balls such that 

\B p {x,r)\ < \B(x,Cr)\ < C d \B(x,r)\, and 

\B(x,r)r\E\ < \B p (x,Cr)nE\ < C p \B p (x, r) n E\. 



Therefore, 



lim l^ r)n g|>lim l^,r)n^l 



r^o \B p (x,r)\ "MC P C d \B(x,r)\ C p C d 
Then since C p , Cd are > 1, r' := c T Cd satisfies < t' < 1/2. Using theorem 12.21 
the proof can then be completed as lemma 5.9.3 in ^B]. □ 



Proof of theorem \4-%% The proof of theorem 5.9.6 in is followed, but one 
must be extremely careful in applying the various results stated thus far in this 
paper. The proof in |16j makes use of the set where the measure-theoretic normal 
exists. This set can be replaced with the reduced boundary without obstacle. A 
proof of a stronger statement which implies (i) can be found in p], but a shorter 
proof is presented here. The complete proof of theorem 14. 21 is given for the reader's 
benefit. 
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By the coarea formula for BVh functions, theorem l2.4l there exists a countable 
dense set Qcl such that |9-At|H(G) < oo and d* H At is countably H-rectifiable for 
t e Q. By lemma 1^71 

S$- 1 [{U(d*, H A t \d* H A t ):teQ}] = 0. 
Using the definitions, one can observe 

(4.1) {x : AO) < t < fj,(x)} C d* tH A t for t E R. 

Thus, F C {Ud* M A t :teQ}, and 5^ _1 [F\{U9| r A t : t € Q}] = 0. Therefore, F 
is countably H-rectifiable. This concludes the proof of (i). 

For (ii), let I = {x : X(x) = oo} U {x : (i{x) = oo}. It be will shown that 
<S^ _1 (i") = 0. Since this is a local question, it may be assumed that u has compact 
support. First it will be shown that S^ 1 ({x : X(x) — oo}) = and S^~ 1 ({P : 
n{P) = — oo}) = so that the set K — {x : (j,(x) — X(x) — oo} is well-defined. Then 
it will be shown that S^ _1 (K) = 0, ending the proof of (ii). By (i), u = fx = A 
Lebesgue almost everywhere. Therefore, letting L t = {x : X(x) > t}, 

\{x : X(x) > t}r\B(x,r)\ _ \{x : u{x) > t}DB(x,r)\ 
\B[x,r)\ ~ \B{x,r)\ ' 

and by definition, X{x) > t implies 

Hm \{x:u(x) <t}nB(x,r)\ = Q 
r->o \B(x,r)\ 

Therefore, D(L t ,x) = 1 for x £ L t . By lemma ECT there exists a countable family 
of gauge-balls, {B p (x i: ri)} such that 

oo oo 

L t C y B p (xi,ri) C [J B(xi, a-i), 

i=l i=l 
oo 

5><) g_1 <C\BLt\ B (G). 

i=l 

Since it has compact support, one may assume diam B(xi, en) < a. From proposi- 
tion and the fact u = X Lebesgue almost everywhere, 

/oo 
\dL t \n(G)dt < oo, 
- OO 

implying 

lim \3LMG) = lim SQ-\d* t HL t ) = 0. 

t-^-oo t— >oo 

Thus, 

OO OO 

S%- X [x : X(x) = oo] = S^-'lf} Lt] < C^inf' 1 < Climinf \dL t \ R (G) = 0. 

1 1 * — ' t — >oo 

t=l i=l 

Let a ->■ 0. Then S 0-1 ({a; : X{x) = oo}) = 0. Similarly, 5 Q_1 ({x : fi(x) = -oo}) = 
0. This concludes the proof of (ii). 
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From part (i), F is H-rectifiable. Using the structure theorem and the implicit 
function theorem, it can be concluded that F is cr-finite with respect to S9~ X [F. 
Thus, 



(jj,-X)dS$~ 1 = / : X(x) < t < n(x)})dt 
Jo 

< [ S^-\d^ H A t )dt (byHU) 
Jo 

/>oo 

</ S^~ X {d* H At)di (by lemma E7|) 

Jo 

poo 

<C \dA t \n(G) dt (bv lemma Oil 

Jo 

< C Var H (u:&) 

< oo. 

Thus, S^iK) = 0. 

Part (iii) will be proved for x G F\{\j{d* tH A t \d* H A t ) : t G Q}. Without loss 
of generality, assume x — 0. One can check using the definitions that for x G 
d* H A t ,D(A t ,x) = 1/2. Now for s < t, A t c A s and 

\A s \AtnB(x,r)\ = \A s nB(x,r)\ - \A t nB(x,r)\ 

implies that D(A s \A t ,x) = 1/2-1/2 = 0. This gives 2M t (0) = ^A s (0) since 

|B(i,r)nA,n5+K(0))| 



r->o |i?(x, r 

£(^ t (0))| + Um JB( 
r™6 |.B(x, r)| r- >o |5(x,r 



]im ,j3(x,r)nA t n5+(^ t (0))| ^ |g(a;,r)n(A B \A t )n5+(^ t (0))| = Q 



and 



lim |(B(^r)\4)n5 H K(0))| = Um |i?(x,r)n{x: M < S } 0^(^(0))! 
r->o \B(x,r)\ r^o \B(x,r)\ 

< Um |B(x,r)n{x: M <t}nS H (^ f (0))| 



r->o |B(x, r) 

= hm — - = 0. 

r-^o \B(x,r)\ 



This concludes the proof of (iii). 

In proving (iv), consider y G F\I, Let e > be such that X(Q) < /u(<3) — e < 
n(Q). By definition, D{A )1 ^ +ei y) = 0, and one can observe that 

lim l^(»Hn5i;K„|_.(Q))n%r)| = i 
r™ |B(y,r)n^(^ (!)) _ e (0))| 

Further, ^( y )_ e C - e,(i(y) + e]}. Therefore, 

| U - 1 {[M(y)-e^(2/) + e]}n5 H (^ M(y) _ £ (0))nB(x,r)| 

hm — = 1. 

|S(s,r)n55(i/A MM _ I (0))| 

Similarly to Droposition l4.il this gives the desired set F~. Similarly, F + is obtained. 

□ 
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5. Defining the trace of an H-BV function on an H-admissible domain 

This section proves an extension and trace theorem for u € BVh{£1), when fl is 
H-admissible. 

Definition 5.1. Let u be a real-valued Lebesgue measurable function defined on fl, 
an open set in G. 

u(x) if x € 0, 
ifxen c . 



u (x) :-- 



Observation: Let F — {x : A„ < ^ M[) }, A t — {x : uq > t}, and Xo £ EC\d H VL be 
such that (hi) of Theorem 14 . 21 applies . Then there is a unit vector v such that 

VA t {xo) = v whenever A„ < t < fj, Uo . 

As in Remark 5.10.6 in one can conclude that 

(5.1) v = ±un(xo), 
and that 

(5.2) if v = Vq(x ), then X Ua (x ) = 0, 

(5.3) if v = -vn(xo), then ^ Ua {x ) = 0. 

Proposition 5.2. If Q is an H-admissible domain and u € BVh{&), then uq £ 
BVh(Gt) and there exist C — C(Q) such that 

I "0 ||sVff(G)< C || U IIbVh(O) ■ 

Proof: It is a fact that u 6 BVh (O) if and only if the positive and negative parts 
of u, u + and u~ , belong to BVh{&)- This fact along with the coarea formula for 
functions in BVh and the structure theorem provide the proof for the theorem 
analogously to lemma 5.10.4 in □ 

Now the trace of a function on dfl for Q an H-admissible domain is given: 

Definition 5.3. If Q is an H-admissible domain and u G BVh(Q), the trace of u 
on dCl, denoted u* , is 

u*{x) := fJ. Ua (x) + X Uo (x), 
where, fi Uo , A Uo are the upper and lower approximate limits of Uq . 

Theorem 5.4. If Q is an H-admissible domain, there is a constant M — M(f2) 
such that 



[ \u*\ d\dn\ H = C{n + m) [ \u*\ dS®' 1 < M 

JG Jd, H n 



whenever u S SVjj(O). 

Proof: The proof follows as theorem 5.10.7 in |16j where the coarea formula for 
BVr is utilized, as is theorem l4.2l (i). □ 
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6. The trace and integral averages 

Throughout this section F is defined as in previous sections and the function U 
is defined as U(x) — (\ u (x) + /j, u (x))/2. The following theorem will be proved: 



Theorem 6.1. Assume u E BVh(G). Then 



(i) lim r ^o j- B {x r) \u - U(x)\G=* dh = for 5j a.e. x G G\F, and 

(ii) for 1 a.e. x E F, there exists a vector v — v{x) such that 



lim + \u - ^(x)]^ dh = 0, 

J B„(x.r)nS-(v) 

and 

lim / \u- \{x)\^t dh = 0. 

The following corollary follows from 15.11 15.21 and 15.31 

Corollary 6.2. Let u E BVh(Q), where ft is H-admissible. Then for 
x E dfl, 

lim / \u{x)-u*{x )\'^ rT dh = Q. 

J B{x ,r)nQ 

The proof of theorem 16.11 relies on the following proposition: 
Proposition 6.3. For every < a < 1, there exists C(a) such that 
\\u\\ Q <C Var H {u:BJx,r)) 

L^(B p (x,r)) 

for all B p (x, r) C G and all u E BVh^ oc (G) such that 

\B p (x,r) n {x : u(x) = 0}| 
\B p (x,r)\ 

Proof. Assume that 



Q 



> a. 



B p (x,r)n{u = Q}[ > a>Q 



\B P (x,r)\ 
Then, 

Hull _Q <\\u— UxrW 9. +\\ u xr\\ Q 

L^ r T(B p (a;,r) ' L^=T (B p (x,r)) ' L^=T (B p (x,r)) 

< CVar u (u;B p (x,r)) + \u x , r \ \B p (x, r)^ 1 ^ . 
The last inequality follows from theorem 12. 31 Next observe, 

K, r | \B p (x,r)\i-^ = \ B ; {x : r JL lim n / \ u \ dh 

|-Bp(a;,0)| 1/Q r ^ Q J B p (x.r) 



\B p (x,r)\VQ J B 



\u\ dh 

,(x,r)n{u^0} 

\ i-i/Q 



-l 

a.e. 



< / \u\Q^ dh 



\B p (x,r)n{u^0}\\ 1/Q 



B p {x.r) J V \ B p{x,r)\ 

< N| o (l-a) 1/Q . 
The conclusion follows. □ 
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Proof of theorem \6. 11 First, observe that replacing B(x,r) by B p (x,r) in the 
definition of fi u (x) and X u (x) result in the same function, since p and d are equiva- 
lent metrics. The proof now follows as theorem 3, section 5.9 in [5]. In this proof, 
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